Methods for processing data samples in communication networks

ABSTRACT

A computer-implemented method for acquiring new data samples and for maintaining a set of data samples in a database, wherein the set of data samples are configured to form input to a function associated with a predictive performance, the method including obtaining at least one relevance metric (M), where the relevance metric is indicative of an increase in the predictive performance of the function when using a data sample as input together with the set of data samples compared to when not using the data sample, obtaining a relevance criterion (C), where the relevance criterion identifies relevant data samples in a set of data samples based on the at least one relevance metric, signaling the at least one relevance metric (M) and the relevance criterion (C) to a data collecting network node.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a 35 U.S.C. § 371 national stage application of PCTInternational Application No. PCT/EP2020/084706 filed on Dec. 4, 2020,the disclosure and content of which is incorporated by reference hereinin its entirety.

TECHNICAL FIELD

The present disclosure relates to methods for storing, modifying and/orsignaling one or more data samples in a communication network. There aredisclosed methods and devices for reducing signaling and data processingoverhead associated with data collection in wireless communicationnetworks. The disclosed methods are particularly suitable for use withmachine learning methods and systems comprising aspects of artificialintelligence, such as neural networks, autoencoder networks and thelike, which often require large amounts of data to function efficiently.

BACKGROUND

Machine learning (ML) is a technique that can be used to find apredictive function for a given dataset; the dataset is typically amapping between a given input to an output. The predictive function (ormapping function) is generated in a training phase, where during thetraining phase it is typically assumed that both the input and outputare known. The test phase then comprises predicting the output for agiven input.

ML applications are becoming ever more popular in radio access networks(RAN) as they allow performance enhancements by utilizing informationavailable in the networks. One such example is secondary carrierprediction (SCP), where information on one or more carriers can be usedto predict the coverage on other frequencies, thus allowing improvedchoices of frequencies for handover, carrier aggregation (CA), idle modeconfiguration, and the like.

Data sample collection is at the heart of AI and ML. Even though cloudsolutions are gaining momentum, there are cases when data needs to bestored locally, for example due to privacy reasons or difficulty/cost oftransporting data. Storing ML models and especially the data samplesrequired for training and operation can however be a problem as thehardware can be very expensive and over dimensioning storage can have asevere impact on the total cost and energy consumption of a predictivefunction.

Finding the most relevant data samples to store and identifying datasamples which can be ignored or even discarded is a challenge. One wouldlike to find the samples that provide the overall best predictiveperformance, by having an accurate representation of the environment. Incase the ML model is trained in a network node separate from the nodethat is generating the data for the model, there is a risk that thesignaling overhead will be prohibitive. This is in particulartroublesome if the data needs to be sent wirelessly from a wirelessdevice to a data processing network node.

There is a need for improved methods for processing data samples incommunication networks.

SUMMARY

It is an object of the present disclosure to provide improved methodsfor processing data samples in communication networks which mitigate atleast some of the above-mentioned issues.

This object is at least in part obtained by a computer-implementedmethod, performed in a data processing network node, for acquiring newdata samples and for maintaining a set of data samples in a database,wherein the set of data samples are configured to form input to afunction associated with a predictive performance The method comprisesobtaining at least one relevance metric M, where the relevance metric isindicative of an increase in the predictive performance of the functionwhen using a data sample as input together with the set of data samplescompared to when not using the data sample. The method also comprisesobtaining a relevance criterion C, where the relevance criterionidentifies relevant data samples in a set of data samples based on theat least one relevance metric. The method further comprises signalingthe relevance criterion to a data collecting network node and receivingone or more data samples from the data collecting network node, wherethe received data samples are associated with relevance metrics thatsatisfy the relevance criterion.

This way the relevance of the reported data samples is increased, atleast on average, which means that the prediction function becomes moreefficient overall. The need to transport data, e.g., across wirelesslinks in a communication network decreases without significantlyimpacting the predictive function performance, which is an advantage.The techniques disclosed herein allow the information in the datasamples to be collected, stored, and processed in a more condensed andefficient way as the most informative parts of the samples areextracted, allowing the gains that higher number of samples provide,such as increased performance by better ML models, while having a lowermemory and/or energy footprint. Communications resources are conservedsince less data samples need to be communicated between the datacollecting network node and the data processing network node. Thesignaling of data for training the various machine learning modelscomprised in a predictive function can be reduced to include the mostinformative samples, this is of high importance especially if the firstnode needs to signal over a wireless channel.

According to some aspects, the method comprises sending the at least onerelevance metric to the data collecting network node. This allows thedata processing network node to also customize the relevance metric,which is an advantage. Alternatively, or in combination, the relevancemetric or parts thereof is pre-configured at the data collecting networknode. This way the data processing network node does not have toconfigure a relevance metric for all data collecting network nodes whichmay be an advantage. Also, the relevance metric can be pre-configured asa default relevance metric to be used at the data collecting networknode, which default metric can be re-configured by the data processingnetwork node, or by some other network node, if required for the dataprocessing task at hand.

According to aspects, the method also comprises training a machinelearning model arranged to indicate the predictive performanceassociated with a data sample in a set of collected data samples,wherein the machine learning model constitutes a relevance metric. Thus,the method allows for relatively advanced forms of relevance metrics,which can be customized in order to increase the performance of the datasample processing.

According to aspects, the method comprises training a machine learningmodel arranged to indicate a novelty metric associated with a datasample in a set of collected data samples, wherein the machine learningmodel constitutes a relevance metric. This ML model can be configured toindicate the novelty of a given data sample in an efficient yet robustmanner. The machine learning model can also be updated based on thereceived one or more data samples, thus keeping the model up to date andmaintaining a high degree of performance by the predictive function. Themethod may furthermore comprise updating the function associated with apredictive performance based on the received one or more data samples,thus keeping the predictive function relevant for the data processingtask at hand.

The at least one relevance metric optionally comprises a distance metricconfigured to quantify a distance between one or more data samples in aset of collected data samples and a set of samples in the database.According to another example, the at least one relevance metriccomprises a reconstruction error metric value obtained from anautoencoder model, and/or a prediction error metric value obtained froma machine learning (ML) model. Thus, it is appreciated that the methodsdisclosed herein are general in the sense that many different types ofrelevance metrics can be used, separately or in combination.

According to aspects, the at least one relevance metric is based on acyclo-stationarity operation configured to indicate a similarity betweenone or more data samples in a set of collected data samples and a set ofbasic patterns. The cyclostationarity operation is particularly suitablefor indicating similarity between captured time-sequences of data and aset of baseline time sequences, as will be explained in more detailbelow. The cyclo-stationarity operation may, for instance, be anauto-correlation function or a cross-correlation function.

According to aspects, the method comprises receiving one or morerelevance metrics associated with the data samples received from thedata collecting network node.

The object is also at least in part obtained by a computer-implementedmethod, performed in a data collecting network node, for acquiring newdata samples and for maintaining a set of data samples in a database ata data processing network node, wherein the set of data samples formsinput to a function associated with a predictive performance. The methodcomprises obtaining at least one relevance metric M, where the relevancemetric is indicative of an increase in the predictive performance of thefunction when using the new data sample as input, and also receiving arelevance criterion C from the data processing network node, where therelevance criterion identifies relevant data samples in a set of datasamples based on the at least one relevance metric. The method alsocomprises collecting a set of collected data samples by the datacollecting network node, selecting one or more relevant data samplesfrom the set of collected data samples based on the relevance metric andon the relevance criterion, where the relevant data samples areassociated with relevance metrics meeting the relevance criterion, andtransmitting the one or more selected data samples to the dataprocessing network node. As discussed above, the at least one relevancemetric can be pre-configured, e.g., as a default relevance metric,and/or it can at least in part be received from the data processingnetwork node.

According to aspects, the relevance metric comprises a machine learningmodel arranged to indicate a predictive performance associated with adata sample in the set of collected data sample. This relevance metricis quite general and versatile and can be used in a wide variety ofdifferent data sample processing operations, which is an advantage.

According to aspects, the selecting comprises any of: adding the one ormore relevant data samples to the database, deleting one or morepreviously collected data samples from the database, increasing a datasample importance metric value associated with a data sample in thedatabase, and/or creating one or more artificial data samples with aspecified data sample importance metric value by combining one or morecollected data samples with one or more data samples in the database.Thus, a wide variety of data sample processing operations are supported,which is an advantage.

According to aspects, the method comprises triggering data sampleselection and transmission based on a trigger criterion comprising anyof a change in mode of operation, a memory status, and/or based on anexternal instruction. This way the data offloading can be efficientlycontrolled, which is an advantage.

According to aspects, the method comprises transmitting a data samplerelevance metric value and/or data sample importance metric value,associated with the selected data samples to the data processing networknode. This sample relevance metric finds multiple uses, as will bediscussed in the following. For instance, it can be used to indicate tothe data processing network node that a given sample is deemed morerelevant compared to others. It can also be used to indicate that somepre-processing has been performed at the data collecting network node.For instance, the method optionally comprises transmitting dataindicative of a number of added and/or deleted data samples of thedatabase to the data processing network node, and/or transmittinginformation indicative of one or more generated artificial data samplesin the database to the data processing network node.

There are also disclosed herein processing units, network nodes, andcomputer program products associated with the above-mentionedadvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will now be described in more detail withreference to the appended drawings, where:

FIG. 1 illustrates an example communication network;

FIGS. 2A and 2B schematically illustrates data sample collection in acommunication network;

FIG. 3 shows an example where the herein disclosed techniques areapplicable;

FIG. 4 illustrates an example of determining a relevance metric;

FIGS. 5,6 are flow charts illustrating methods;

FIG. 7 shows an example wireless device;

FIG. 8 schematically illustrates a communications network;

FIG. 9 schematically illustrates processing circuitry; and

FIG. 10 shows a computer program product;

DETAILED DESCRIPTION

Aspects of the present disclosure will now be described more fullyhereinafter with reference to the accompanying drawings. The differentdevices, systems, computer programs and methods disclosed herein can,however, be realized in many different forms and should not be construedas being limited to the aspects set forth herein. Like numbers in thedrawings refer to like elements throughout.

The terminology used herein is for describing aspects of the disclosureonly and is not intended to limit the invention. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise.

The concept of a data sample is herein given a broad interpretation. Adata sample can, for instance, be defined as a set of features with orwithout a target variable in a certain radio network operation. Forexample, in case of SCP, the sample could include the features (sourcecarrier measurements), and the target (secondary carrier measurements).A data sample can also be a piece of data together with an associateddata relevance metric, or just a measurement value of some parameter ofinterest. A time series and/or frequency series of collected data mayalso be referred to as a data sample herein.

FIG. 1 illustrates an example communication network 100 where accesspoints 110, 110′ provide wireless network access to wireless devices140, 140′ over a coverage area 111. An access point in a fourthgeneration (4G) 3GPP network is normally referred to as an evolved nodeB (eNodeB), while the access points in a fifth generation (5G) 3GPPnetwork are often referred to as a next generation node Bs (gNodeB). Theaccess points 110, 110′ are connected to some type of core network 120,such as an evolved packet core network (EPC). The EPC is an example of anetwork which may comprise wired communication links, such as opticallinks 121, 121′. Asymmetric digital subscriber line (ADSL) communicationnetworks 122 constitute another example or a wired communicationsnetwork. ADSL may, e.g., be used to connect stationary users 150 to thecore network 120.

The wireless access network 100 supports at least one radio accesstechnology (RAT) for communicating 145, 145′ with wireless devices 140,140′. It is appreciated that the present disclosure is not limited toany particular type of wireless access network type or standard, nor anyparticular RAT. The techniques disclosed herein are, however,particularly suitable for use with 3GPP defined wireless accessnetworks.

Radio communication 145, 145′ takes place over a radio propagationchannel. A radio propagation channel comprises a physical transmissionmedium and normally introduces one or more forms of distortion to thetransmitted information. It is of interest to collect data samplesassociated with such impairments, in order to be able to predict futureimpairments that will affect users in similar communication scenarios.

FIG. 7 illustrates an example wireless device 700, this device mayaccording to one example constitute the data collecting network node.

FIG. 8 illustrates a data processing network node 800, here exemplifiedas a central baseband unit (BBU). This node may of course also berealized as a virtual node or part of some cloud-based function asillustrated in FIG. 8 .

There is an ongoing discussion in 3GPP on how to support ArtificialIntelligence (AI) and Machine Learning (ML), see, e.g., the proposedstudy item description in RP-201304 “New SID: Study on furtherenhancements for data collection”, RAN #88-e meeting, revision ofRP-200770. The scope of the discussion is to study high level principlesfor RAN intelligence enabled by AI, the functional framework (e.g. theAI functionality and the input/output of the component for AI enabledoptimization) and identify the benefits of AI enabled NG-RAN throughpossible use cases e.g. energy saving, load balancing, mobilitymanagement, coverage optimization.

It is appreciated that this type of processing may require largequantities of data in order to be really effective, and transportingthis data over the network, in particular over bandwidth constrainedwireless links, may result in prohibitive levels of signaling overhead.

The present disclosure evolves around methods and a framework for savingand adjusting collection of data samples for use in some form ofML-based operation. The method helps out in identifying relevant datasamples that should be combined and stored for use in the ML processing.One of the key concepts disclosed herein is for a data processing nodeto instruct a data collecting node on how to collect data, andoptionally also when to off-load the collected data. This way the dataprocessing node can tailor the data sample reporting from different datacollecting nodes to best suit the application and processing resourcesat hand. Different instructions can be issued for different MLoperations at the data processing network node, and differentinstructions can also be issued to different types of data collectingnetwork nodes, or data collecting network nodes in differentcommunication scenarios, such as at different spatial locations orcommunicating in different frequency bands.

FIG. 2A schematically illustrates a system 200 for collecting datacomprising a data processing network node 210 and a data collectingnetwork node 220. The data processing network node 210, and optionallyalso the data collecting network node 220, comprises a database 215 withdata samples for some data processing task. A number of potentially“collectible” and “reportable” data samples 240 are conceptually shownin FIG. 2A. The data collecting network node 220 collects, i.e.,samples, a subset 225 of the available data, and selects one or more ofthese collected data samples for reporting 230 to the data processingnetwork node 210. It is appreciated that the two nodes 210, 220 may bephysically collocated as modules of a single network node 250.

The data processing node sends a relevance metric M and a relevancecriterion C to the data collecting network node 220. The relevancemetric may also be pre-configured at the data collecting network node,e.g., as a default relevance metric to be used in case the dataprocessing network node does not explicitly provide a metric. The datacollecting network node may also store a number of different metrics,and the data processing network node can indicate which one to use for agiven data collecting task. In general, a relevance metric M defines howrelevance is to be measured for a data sample 240, while the relevancecriterion C defines what constitutes a relevant data sample. Examples ofthis relevance metric will be discussed in more detail below. Havingknowledge of the metric M and the criterion C enables the datacollecting node to collect (and optionally also to pre-process) datasamples 225 according to the instructions of the data processing networknode 210. Thus, the data samples 230 reported back to the dataprocessing network node are efficiently reported, where efficiency isdefined by the system 200, and may be application specific as well astime-varying. For instance, efficiency may be measured in terms ofconsumed communication resources such as bandwidth and/or time,reporting delay, and/or in terms of the performance of the dataprocessing operation at hand. One common data processing operation issome form of prediction based on measured data. Different relevancemetrics M and relevance criteria C may be issued to different datacollection network nodes, and several metrics and criteria may also beissued to a single data collecting network node, for instance in casethe data collecting network node collects data samples for more than onepredictive function.

An example of a predictive function is given in FIG. 3 . A firstwireless device 310 and a second wireless device 320 measures ReferenceSignal Receive Power (RSRP) measurements on one or more carriers asfunction of device location. A predictive function such as an AI modelor ML model can be trained for example to use a time-series of RSRPmeasurements to predict a future signal quality. WO2020226542 A1provides background material and discusses the training of these typesof systems. The techniques disclosed in WO2020226542 A1 are at leastpartly applicable also to the examples disclosed herein. In the example300, the two wireless devices are turning around the same corner 330according to the location plot and experience a drop in signal strengthwhen doing so. The first wireless device 310 first turns around thecorner and experiences a large signal quality drop. The idea is then touse the predictive function to predict the drop in signal power at thesecond wireless device 320, and mitigate this drop, e.g., by a change ofcarrier or adjustment in communications resource assignment like theassignment of a stronger channel code. According to the currentlyproposed methods, the network can, based on the time-series of RSRPdata, use the pattern similarities to take a decision on how to handleany new RSRP time-series data sample measurements. For example, it maybe sufficient to use one set of data from one of the wireless devicessince the time-series data sets if the RSRP data sets are similar. Inother words, if one wireless device has reported a measurement for agiven path, then another wireless device need not report the samemeasurement, as this would not contribute with very much additionalinformation to the already existing data samples in the database 215.

FIG. 5 is a flow chart illustrating one such computer-implementedmethod, performed in a data processing network node 210, for acquiringnew data samples 230 and for maintaining a set of data samples in adatabase 215, wherein the set of data samples are configured to forminput to a function associated with a predictive performance The methodcomprises obtaining Sa1 at least one relevance metric M, where therelevance metric is indicative of an increase in the predictiveperformance of the function when using a data sample as input togetherwith the set of data samples 215 compared to when not using the datasample. The method also comprises obtaining Sa2 a relevance criterion C,where the relevance criterion identifies relevant data samples in a setof data samples based on the at least one relevance metric. The methodfurther comprises signaling Sa3 the relevance criterion C to a datacollecting network node 110, 120, 130, 140, 150, 220, and receiving Sa4one or more data samples 230 from the data collecting network node,where the received data samples are associated with relevance metrics Mthat satisfy the relevance criterion C.

The at least one relevance metric M may according to one example besignaled Sa34 to the data collecting network node, or it can bepre-configured Sa35 at the data collecting network node. It isfurthermore noted that the signaling of the relevance criterion C to thedata collecting network node may be performed via some other networknode. For instance, a configuration network node 260 may be in charge ofparameterizing, i.e., setting up, the data processing operation at thedata collecting network node and at the data processing network node.This configuration network node 260 then signals the relevance metric Mand/or the relevance criterion C to the data collecting network nodeand/or to the data processing network node, as schematically illustratedin FIG. 2B. It is furthermore appreciated that any of the datacollecting network node, the configuration network node, and the dataprocessing network node may be co-located on the same physical entity,or they can be implemented on two physically separate entities, or theycan be distributed over more than two entities.

FIG. 6 is a flow chart illustrating a corresponding computer-implementedmethod for acquiring new data samples 230 and for maintaining a set ofdata samples in a database 215 is performed in the data collectingnetwork node 220. The method comprises obtaining Sb1 at least onerelevance metric M, where the relevance metric is indicative of anincrease in the predictive performance of the function when using thenew data sample as input. The method also comprises obtaining Sb2 arelevance criterion C from the data processing network node or from aconfiguration network node 260, where the relevance criterion identifiesrelevant data samples in a set of data samples based on the at least onerelevance metric. The method further comprises collecting Sb3 a set ofcollected data samples 225 by the data collecting network node,selecting Sb4 one or more relevant data samples from the set ofcollected data samples based on the relevance metric M and on therelevance criterion C, where the relevant data samples are associatedwith relevance metrics meeting the relevance criterion, and transmittingSb5 the one or more selected data samples 230 to the data processingnetwork node.

The at least one relevance metric M can be pre-configured Sb11 at thedata collecting network node, and/or at least in part received Sb12 fromthe data processing network node. Of course, the relevance criterion mayalso be received from some other network node configured to parameterizethe data sample collection process.

It is appreciated that the methods disclosed herein can also beperformed in any of the network nodes 110, 120, 130, 140, 150 discussedabove in connection to FIG. 1 . The two methods of FIG. 5 and FIG. 6 canalso of course be performed in the same network node 250 as mentionedabove in connection to FIG. 2A, in which case efficient data collectionand processing is enabled in a single node.

The relevance metric M may, as discussed above, take on many forms, andcan be different for different applications. The method may, forinstance, comprise training Sa11 a machine learning model arranged toindicate the predictive performance associated with a data sample in aset of collected data samples 225. In this case the relevance metric Mcomprises the machine learning model itself. For example, one can use atrained model to identify or determine the novelty of a given datasample based on the prediction error of the model. For example, in casethe data-processing node receives a sample (x,y), where also the modelcan predict an estimate of y based on x. The novelty of the sample wouldthen be indicated by the model error, i.e., the difference between thetrue value of y and its estimate. In one example, this can compriseperforming an inter-frequency measurement based on predicted goodcoverage, using source carrier measurements (x). Next, when the UEactually performs the measurement, it receives y and can evaluate howgood the model is. This means that the machine learning model iscommunicated in some way to the data collecting network node. This canbe performed, e.g., by signaling which model out of a set of pre-trainedmodels to use when collecting data of a given type, or a new model canbe transmitted to the data collecting network node.

The decision on how to rate (in terms of importance) a new data samplefor a given existing dataset can be based on a function which outputs adistance metric. Thus, optionally, at least one relevance metric Mcomprises a distance metric d configured to quantify a distance betweenone or more data samples in a set of collected data samples and a set ofsamples in the database.

One such example function is the weighted Euclidian distance compared tothe other samples in the database 215. The weighted Euclidian distanced, this can be calculated for n features by:

${d = \sqrt{\sum\limits_{i = 1}^{n}{w_{i}\left( {x_{a} - x} \right)}^{2}}},{d \in D}$

Another type of decision criterion could be based on some of theavailable distance metrics in the literature. Some non-limiting examplesinclude Manhattan (L1-norm), Euclidean (L2-norm), Minkowski, Cosine andChebychev type of distance metrics.

In case of supervised learning, the data collecting network node 220compares the distance d with the samples within its own target class. Inthe literature there are many techniques that can use the distance toreach a decision on the importance of a sample and this disclosure doesnot exclude one of them. Some non-limiting examples includedensity-based approaches, proximity approaches (maximum distance toother points, average distance to other points etc.) or in case ofregression, only the samples within a range of its target regressionvariable are being used. Another alternative will be to pickmeasurements that do not reduce the overall goodness of the fit.

Another ML-based decision criterion would be to train a model todirectly identify or determine the novelty of a given data sample. Inthis case, the decision is not based on a distance metric or the like,as previously described, but the model would indicate “how novel” thecurrent data sample is. This approach voids the need to store a largenumber of samples to calculate point-to-point distances or a set ofpatterns, and only a single neural network would be required which is anadvantage. This approach can be enabled using Random NetworkDistillation (RND) to train the model. Furthermore, this solution couldeither be associated with a dataset (trained with all points on thedataset to identify the novel samples) or with specific deployed models(trained with the same samples used during the training of that specificmodel) in this case, the network would identify data samples that areunknown to that specific model. The training of such a model could bedone either periodically over the new data set, or in the case of an RNDdirected associated with a model, it would be trained in parallel withthat model. RND methods are discussed, e.g., in the article “EXPLORATIONBY RANDOM NETWORK DISTILLATION” by Yuri Burda, Harrison Edwards, AmosStorkey, and Oleg Klimov, 30 Oct. 2018, arXiv:1810.12894v1.

Thus, in other words, the methods may also comprise training Sa12 amachine learning model arranged to indicate a novelty metric associatedwith a data sample in a set of collected data samples 225. The machinelearning model then constitutes the relevance metric M and may also besaid to constitute the relevance criterion C since it also determines ifa data sample is relevant or not for the data processing task at hand.According to some aspects, the method may also comprise updating Sa13the machine learning model based on the received one or more datasamples 230. The method may further comprise updating Sa14 the functionassociated with a predictive performance based on the received one ormore data samples 230. The relevance metric M may furthermore comprise areconstruction error metric value obtained from an autoencoder model.The autoencoder can be used to represent the current samples at the dataprocessing network node, and a high reconstruction error for anyobtained samples at the collecting nodes then indicate a novel, i.e.,important, sample. High reconstruction error corresponds to a largerelevance metric.

According to other aspects, the at least one relevance metric Mcomprises a prediction error metric value obtained from a machinelearning, ML, model. For example, the model can forecast future signalquality values, and the data collecting network node can compare theforecasted value with the actual measured and include those time-seriesof samples that generated a high prediction error. The relevance metricis hence related to the machine learning model prediction error. In thiscase, the function associated with a predictive performance is same asthe ML model used to determine the relevance metric.

According to other aspects, with particular reference to FIG. 4 , the atleast one relevance metric M can be based on a cyclo-stationarityoperation 410 configured to indicate a similarity between one or moredata samples in a set of collected data samples a set of basic patternsstored in the database or in some other test set and. This decisioncriterion for classifying data samples in terms of importance orrelevance considers a group of samples (e.g., in spatial domain and/orin frequency domain) or series of samples (in-time domain). A node,normally the data collecting network node 220 conducts acyclostationarity operation on a group of samples (referred to herein asa test pattern) against a number of pre-determined existing basicpatterns (which are stored at the node), as illustrated in FIG. 4 . Thecyclo-stationarity operation can be an auto-correlation function or across-correlation function. The basic patterns are consideredindividually for each period of time or geographical location samples.For instance, for time t₀ to t_(N) we have basic patterns 430 of P_(n0)to P_(nX), whereas for time t_(N+1) to t_(M) we have basic patter ofP_(n0) to P_(nY)

For instance, cyclostationarity similarity between patterns can becalculated as the auto-correlation between the existing pattern ‘x’ andthe pattern to be tested ‘y’.

${S_{x,y}^{T}(t)} = {\lim\limits_{T\rightarrow\infty}{\frac{1}{T}{E\left\lbrack {{R_{x}(t)}{R_{y}\left( {t - T} \right)}} \right\rbrack}}}$

High similarity increases S_(x,y) ^(T)(t), see, e.g., example 440 whilelow similarity reduces S_(x,y) ^(T)(t), see, e.g., example 450 in FIG. 4. This relevance metric can be compared to a relevance criterion,indicated as γ in FIG. 4 , and a decision on a course of action for agiven data sample is made based on a comparison of the relevance metricvalue to the relevance criterion.

A node (or network) may define new patterns (to be considered as a newbasic pattern) if the corresponding eigenvector shows full or partialorthogonality to existing basic patterns. If a data sample is found tohave a high degree of similarity when compared to the basic patterns, itmay not be deemed relevant for the task at hand and can therefore beignored. Having observed more than one “instance” of a given pattern canbe remembered, e.g., by increasing a weight parameter associated withthe basic pattern. Each basic pattern can also be associated with afrequency variable indicating the statistics of the determinedcyclo-stationarity measures.

The methods disclosed herein may furthermore comprise signaling Sa31 theat least one relevance metric M and the relevance criterion C, over aPhysical Downlink Control Channel (PDCCH) of a third generationpartnership program (3GPP) communications network. According to otheraspects, the method comprises signaling Sa32 the at least one relevancemetric M and the relevance criterion C as a Radio Resource Control (RRC)reconfiguration message of a 3GPP communications network or signalingSa33 the at least one relevance metric M and the relevance criterion Cas a dedicated message over a Physical Downlink Shared Channel (PDSCH)of a 3GPP communications network. These communications channels areresource constrained. The methods proposed herein reduce thecommunication requirements in terms of bits/sec, which is an advantage.

It is noted that the data collecting network node 220 may also signaladditional information in addition to the selected samples deemed asrelevant. For instance, the method optionally comprises receiving Sa41one or more relevance metrics associated with the data samples 230received from the data collecting network node. A relevance metric mayoptionally comprise a machine learning model arranged to indicate apredictive performance associated with a data sample in the set ofcollected data samples 225.

The selecting operation may be performed in a number of different ways,which ways can of course also be combined for additional advantages. Forinstance, the selecting may comprise adding Sb41 the one or morerelevant data samples to the database 215, deleting Sb42 one or morepreviously collected data samples from the database 215, and/orincreasing Sb43 a data sample importance metric value associated with adata sample in the database 215.

In one embodiment, a weight or importance value is stored for eachsample in the dataset, i.e., the dataset consists of tuples ([[x₁, w₁];[x₂, w₂], . . . , [.x_(n), w_(n)]]) where x_(i) is the i-th data sampleand w_(i) is its corresponding weight or importance value. The weight ofa sample can be increased if a new sample is the closest in terms of acertain distance criterion. For example, the method may compriseincreasing the value of w₁ if a new sample is close in some distancemeasure to x₁ of the database samples, or at least within some thresholdrange of x₁. Consequently, the herein disclosed methods may comprisetransmitting Sb51 a data sample relevance metric value and/or datasample importance metric value, associated with the selected datasamples 230 to the data processing network node.

The methods disclosed herein may furthermore comprise transmitting Sb52data indicative of a number of added and/or deleted data samples of thedatabase to the data processing network node, as well as transmittingSb53 information indicative of one or more generated artificial datasamples in the database to the data processing network node. This meansthat the weight parameter can be extended to include also additionalitems of information. The data processing network node 210 may requestthis information, and the data collecting network node may respond if ithas the ability to do so. For instance, the data collecting network node220 may adapt its data collecting processes to accommodate requestsreceived from the data processing network node 210.

The sample weights discussed above can have an advantageous effect onthe model training, for example by including the sample weights in theoptimization function of the predictive function. A sample with highweight may be interpreted as, e.g., more frequently occurring, and byadding higher importance to a prediction that will be more frequentlyperformed, the overall prediction performance tends to increase. Atypical optimization is to minimize the mean squared error of the modeloutput and the true value. I.e.

${MSE} = {\frac{1}{N}{\Sigma\left( {{f(x)} - y_{true}} \right)}^{2}}$

where y_(true) is the desired output for a given input x, ƒ(x) is theoutput of the predictive function and where the MSE is calculated forall stored N samples. A sample weight w_(s) can be included by addingthe sample weight as a factor:

${MSE_{w}} = {\frac{1}{N}{\sum\limits_{s}{w_{s}\left( {{f\left( x_{s} \right)} - y_{true_{-}s}} \right)}^{2}}}$

where the weighted MSE is calculated for all stored N samples.

The data collecting network node (and potentially also the dataprocessing network node) can also perform clustering metrics such ask-means or the like to reduce the database to k samples. The datacollecting network node can according to one example be configured witha clustering method to reduce its database of samples. As a non-limitingexample, the node can be configured to produce k-samples using thek-means method. Other examples can be clustering, self-organizing mapsand principal component analysis. Thus, the selecting may comprisecreating Sb44 one or more artificial data samples with a specified datasample importance metric value by combining one or more collected datasamples with one or more data samples in the database. A flag orindicator variable can of course be added to the data samples whichindicates which data samples were generated artificially.

In the literature known techniques that can be used for sample(s)generation and provided here as non-limiting examples includeprobabilistic models, classification-based imputation models, andgenerative adversarial neural networks. Different quality metrics aretypically used to rate the performance of the above techniques that canbe used to assess which technique is more appropriate for the specificuse-case.

In case the database can only store a certain number of N samples (or awireless device can only report N samples), and the new sample(s) givesa total population of N+X samples then the network has to discard ordelete X samples to keep the population to the size of N. The selectionof which points to be picked (N) or be deleted (X) can be done accordingto different methods. For picking N number of samples where the summedEuclidian distance of all sample-pairs is maximized can be used. Anothermethod is to use the aforementioned the ML model prediction if presentas a decision criterion, to remove samples that already have an accurateprediction. In general, the methods disclosed herein may also comprisetriggering Sb40 data sample selection and transmission based on atrigger criterion comprising any of a change in mode of operation (suchas a wireless device entering an idle mode of operation), a memorystatus, and/or based on an external instruction (perhaps received fromthe data processing network node if this node is in need of additionaldata for some given predictive task at hand). Alternatively, for findingthe points X that need to be deleted goodness of fit, confidenceinterval, or Kolmogorov-Smirnov tests and techniques are typically used.

FIG. 9 schematically illustrates a device 900 comprising a number offunctional units. The device 900 embodies a network node 110, 120, 130,140, 150, 210 according to embodiments of the discussions herein, interms of functional units of the device 900. Processing circuitry 910 isprovided using any combination of one or more of a suitable centralprocessing unit CPU, multiprocessor, microcontroller, digital signalprocessor DSP, etc., capable of executing software instructions storedin a computer program product, e.g. in the form of a storage medium 930.The processing circuitry 910 may further be provided as at least oneapplication specific integrated circuit ASIC, or field programmable gatearray FPGA.

Particularly, the processing circuitry 910 is configured to cause thedevice 900 to perform a set of operations, or steps, such as the methodsdiscussed in connection to FIG. 8 and the discussions above. Forexample, the storage medium 930 may store the set of operations, and theprocessing circuitry 910 may be configured to retrieve the set ofoperations from the storage medium 930 to cause the device to performthe set of operations. The set of operations may be provided as a set ofexecutable instructions. Thus, the processing circuitry 910 is therebyarranged to execute methods as herein disclosed. In other words, thereis shown a network node 110, 120, 130, 140, 150, 210 comprisingprocessing circuitry 910, a network interface 920 coupled to theprocessing circuitry 910 and a memory 930 coupled to the processingcircuitry 910, wherein the memory comprises machine readable computerprogram instructions that, when executed by the processing circuitry,causes the network node to perform operations as discussed herein.

The storage medium 930 may also comprise persistent storage, which, forexample, can be any single one or combination of magnetic memory,optical memory, solid state memory or even remotely mounted memory.

The device 110, 120, 130, 140, 150, 210 may further comprise aninterface 920 for communications with at least one external device. Assuch the interface 920 may comprise one or more transmitters andreceivers, comprising analogue and digital components and a suitablenumber of ports for wireline or wireless communication.

The processing circuitry 910 controls the general operation of thedevice 110, 120, 130, 140, 150, 210, e.g., by sending data and controlsignals to the interface 920 and the storage medium 930, by receivingdata and reports from the interface 920, and by retrieving data andinstructions from the storage medium 930. Other components, as well asthe related functionality, of the control node are omitted in order notto obscure the concepts presented herein.

FIG. 10 illustrates a computer readable medium 1010 carrying a computerprogram comprising program code means 1020 for performing the methodsillustrated in, e.g., FIGS. 5 and 6 , when said program product is runon a computer. The computer readable medium and the code means maytogether form a computer program product 1000.

1. A computer-implemented method, performed in a data processing networknode, for acquiring new data samples and for maintaining a set of datasamples in a database, wherein the set of data samples are configured toform input to a function associated with a predictive performance, themethod comprising obtaining (Sa1) at least one relevance metric (M),where the relevance metric is indicative of an increase in thepredictive performance of the function when using a data sample as inputtogether with the set of data samples compared to when not using thedata sample, obtaining (Sa2) a relevance criterion (C), where therelevance criterion identifies relevant data samples in a set of datasamples based on the at least one relevance metric, signaling (Sa3) therelevance criterion (C) to a data collecting network node, and receiving(Sa4) one or more data samples from the data collecting network node,where the received data samples are associated with relevance metrics(M) that satisfy the relevance criterion (C).
 2. The method according toclaim 1, comprising signaling (Sa34) the at least one relevance metric(M) to the data collecting network node.
 3. The method according toclaim 1, wherein the relevance metric (M) is pre-configured (Sa35) atthe data collecting network node.
 4. The method according to claim 1,comprising training (Sa11) a machine learning model arranged to indicatethe predictive performance associated with a data sample in a set ofcollected data samples, wherein the machine learning model constitutes arelevance metric (M).
 5. The method according to claim 1, comprisingtraining (Sa12) a machine learning model arranged to indicate a noveltymetric associated with a data sample in a set of collected data samples,wherein the machine learning model constitutes a relevance metric (M).6. The method according to claim 4, comprising updating (Sa13) themachine learning model based on the received one or more data samples.7. The method according to claim 1, comprising updating (Sa14) thefunction associated with a predictive performance based on the receivedone or more data samples.
 8. The method according to claim 1, whereinthe at least one relevance metric (M) comprises a distance metric (d)configured to quantify a distance between one or more data samples in aset of collected data samples and a set of samples in the database. 9.The method according to claim 1, wherein the at least one relevancemetric (M) comprises a reconstruction error metric value obtained froman autoencoder model.
 10. The method according to claim 1, wherein theat least one relevance metric (M) comprises a prediction error metricvalue obtained from a machine learning, ML, model.
 11. The methodaccording to claim 1, wherein the at least one relevance metric (M) isbased on a cyclo-stationarity operation configured to indicate asimilarity between one or more data samples in a set of collected datasamples and a set of basic patterns.
 12. The method according to claim11, wherein the cyclo-stationarity operation is an auto-correlationfunction or a cross-correlation function.
 13. The method according toclaim 1, comprising signaling (Sa31) the at least one relevance metric(M) and the relevance criterion (C) over a Physical Downlink ControlChannel, PDCCH, of a third generation partnership program, 3GPP,communications network.
 14. The method according to claim 1, comprisingsignaling (Sa32) the at least one relevance metric (M) and the relevancecriterion (C) as a Radio Resource Control, RRC, reconfiguration messageof a 3GPP communications network.
 15. The method according to claim 1,comprising signaling (Sa33) the at least one relevance metric (M) andthe relevance criterion (C) as a dedicated message over a PhysicalDownlink Shared Channel, PDSCH, of a 3GPP communications network. 16.The method according to claim 1, comprising receiving (Sa41) one or morerelevance metrics associated with the data samples (230) received fromthe data collecting network node.
 17. A computer program comprisingprogram code means for performing a method according to claim 1 whensaid program is run on a computer or on processing circuitry of anetwork node.
 18. A computer program product comprising a computerprogram according to claim 17, and a computer readable means on whichthe computer program is stored.
 19. (canceled)
 20. Acomputer-implemented method, performed in a data collecting networknode, for acquiring new data samples and for maintaining a set of datasamples in a database at a data processing network node, wherein the setof data samples forms input to a function associated with a predictiveperformance, the method comprising obtaining (Sb1) at least onerelevance metric (M), where the relevance metric is indicative of anincrease in the predictive performance of the function when using thenew data sample as input, obtaining (Sb2) a relevance criterion (C) fromthe data processing network node, where the relevance criterionidentifies relevant data samples in a set of data samples based on theat least one relevance metric, collecting (Sb3) a set of collected datasamples by the data collecting network node, selecting (Sb4) one or morerelevant data samples from the set of collected data samples based onthe relevance metric (M) and on the relevance criterion (C), where therelevant data samples are associated with relevance metrics meeting therelevance criterion, and transmitting (Sb5) the one or more selecteddata samples to the data processing network node. 21-32. (canceled) 33.A network node, comprising: processing circuitry; a network interfacecoupled to the processing circuitry; and a memory coupled to theprocessing circuitry, wherein the memory comprises machine readablecomputer program instructions that, when executed by the processingcircuitry, causes the network node to perform a method according toclaim
 20. 34. (canceled)